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(54) PATTERN RECOGNIZING/COLLATING DEVICE 

(57)Abstract 

PROBLEM TO BE SOLVED: To accurately perform a collation to a 
person even about any face photograph by previously obtaining a 
conversion to make a model and a fluctuation amount be orthogonal 
to each other from many face pictures and face photographs, 
extracting a characteristic using this conversion and collating a face 
not attaching to any group of models to a photograph. 
SOLUTION: A covariance calculating unit 3 fetches picture data 
fi'om an input pattern input means 1 and a model pattern input 
means 1. and calculates a covariance matrix as regarding one 
picture data as a vector. A characteristic extracting unit 4 
calculates a conversion matrix from the covariance matrix, and at 
the same time, performs a vector conversion as regarding one 
picture data as a vector and calculates a characteristic vector. 
Then, the selection of a characteristic space in that the space of 
the model pattern and the space of modelHnput varied vector are 
orthogonal is performed, and the collating processing of the face 
picture and the face photograph by the input pattern input means 1. 
the model pattern input means 2. the covariance calculating unit 3 
and the determination unit 5. 
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♦ NOTICES ♦ 

JPO and HC\P\ are not responsible for any 
damages caused by the use of this translation. 

1. This document has been translated by computer. So the translation may not reflect the original precisely. 

2. *^p¥* shows the word which can not be translated. 
3.1n the drawings, any words are not translated. 
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[Claim(s)] 

[Claim 1] A model pattern input means to input the model pattern M (for it to also be called model ** KUTORU). An 
input configuration input means to input input configuration I for recognition (for it to also be called an input vector). 
A model vector covariance input means to input the covariance matrix Cm of a model vector, A model-input 
fluctuation covariance Input means to make learn beforehand the covariance matrix Cp of the fluctuation to the 
input configuration which corresponds from each model pattern, and to input it. Weighted average with the model- 
input fluctuation covariance matrix inputted from the model vector covariance matrix inputted from said mode! 
vector covariance input means, and said model-input fluctuation covariance input means Cs^^alpha Cm-Kl "alpha) 
Cp (alpha is the real number of 0< alpha<1) ... (1) 

Matrix Cs of the output of a covariance weighted average generation means to be alike, to follow and take and to 
newly generate Matrix Cs, and said covariance weighted average generation means Cs= (AQ 1/2) (Q1/2AT) ... (2) 
(A is the normalization characteristic vector matrix of Cs) 

(Diagonal matrix which consists of characteristic value to which Q corresponds) 

( — Q — the square root matrix of 1/2Q. and AT — transposed-matrix) of A — like — spectral decomposition — 
carrying out — this — matrix D**Q-1 / 2AT (Q-1/2 are the inverse matrix of the square root matrix of Matrix Q) 
The diagonalization means of 1st and matrix DCmDT which changed the model pattern covariance matrix 

Cm according to Matrix D DCmDT=BPBT ... (3) 
(B is the normalization characteristic vector matrix of DCmDT) 

output Q-1/2 of the 2nd diagonalization means which carries out spectral decomposition like (the diagonal matrix 
which consists of characteristic value to which P corresponds), and acquires Matrix B, and said 1st and 2nd 
diagonalization means — AT and B — using — H**WBTQ-1/2AT ... (4) 

(W**diag (alphal, alpha2. ... alphan) and (non-negative-number) with alpha i [ suitable ]) are followed. Matrix H is 
generated and held, and it is from the model pattern M and input configuration 1 to the run time of recognition. 
M'**HM. r**HI ... (5) 

alike — following — each — a feature vector — M — ' — I — ' — extracting — a feature extraction — a means - 

- said — a feature extraction — a means — having extracted — an input configuration — I — a feature vector — I 

— • — a model — a pattern — M — a feature vector — M — ' — distance — ||M'-r || (||*|| is Euclidean 
distance) ... (6) 

The pattern recognition and the collating unit characterized by providing a judgment means to judge to which model 
an input configuration corresponds by this by finding out the model pattern with which also has a small feature 
vector (recognition). 

[Claim 2] Similarity of the feature vector of a model pattern and an input configuration (M -D /(|Mi|r|) ... (7) 
For the inner product of a vector, and |*|, ((* and *) are the pattern recognition and a collating unit according to 
claim 1 characterized by providing a judgment means to evaluate according to magnitude-of-a-vector) and to judge 
whether this value of that input configuration and model is the same by whether it is more than constant value. 



[Translation done.] 
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* NOTICES * 

JPO and NCIPI are not responsible for any 
damages caused by the use of this translation. 

I.This document has been translated by computer. So the translation may not reflect the original precisely. 
2j¥*s¥^ shows the word which can not be translated. 
3.1n the drawings, any words are not translated. 

DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Field of the Invention] This invention relates to the pattern recognition and the collating unit which recognizes and 
judges a fluctuation component especially in the space changed so that it might intersect perpendicularly with a 
model component in the pattern recognition and the collating unit for authentication of ID system using a person 
face, and a license pocket person, a man machine interface, or security about the pattern recognition and the 
collating unit used for the information compression of the user identification which used the photograph of his face 
etc., or a low bit rate communication link. 
[0002] 

[Description of the Prior Art] Based on the secondary statistic (covariance) calculated from the model set 
registered into the database, in distribution of a pattern, i.e., the data space mentioned above, the method which 
attracts attention recently in the technical field of pattern recognition (for example, face image recognition and 
speech recognition) presumes the part which the set of a pattern occupies, and performs the feature extraction 
from a pattern based on this. For example, KL (Karhunen-Loeve) expansion method learned well performs a feature 
extraction by KL expansion, and is reference. M.Turk &: [ A.Pentland] "Face Recognition Using Elgenfaces" 
Proceedings of IEEE, CVPR91. It is stated in detail and there are many to which other approaches applied to this 
correspondingly. 

[0003] With the KL method, they are the model image M and input configuration I. M=sigmagammaiEi 
I=sigmagammai'Ei ... (8) 

(For gammai, i component of M and gammai' are i component of I) 

It approximates by the linear combination of p base vectors Ei (i= 1 ... p), and collating is taken between 
approximation data as shown for (taking the sum about i= 1...p). 

[0004] The characteristic vector corresponding to p things (for example, about 100 pieces) is used for the KL 
method from what has the large characteristic value of the covariance matrix obtained from w instruction pattern 
data as this base vector. If the base vector constitutes space, it dissociates best namely, it will be easy to 
distinguish and the instruction data by which (1) projection was carried out will become. 

(2) The noise contained in a pattern can take and remove the component (fluctuation) which appears irregularly. 
It is thought that it has the advantage to say. The point which it should be careful of in this KL method is a point 
which assumes that distribution of the pattern vector presumed based on the statistic obtained from a model 
pattern set has generality, i.e.. it is about applied also in the pattern of an input 

[0005] For example, in face image recognition, when the fluctuation from the model of an input configuration is not 
so large, also experimentally, it is actually checked that a recognition rate with a very high precision is attained. 
[0006] 

[Problem(s) to be Solved by the Invention] However, in the above conventional methods, when the difference 
between an input configuration and a model became large, there was a problem that sufficient recognition rate could 
not be offered. This is a serious problem which generates when environments, such as lighting conditions at the time 
of photography, change a lot by the input image and the model image in image recognition, and is often generated 
actually. 

[0007] The cause of the above-mentioned problem in a conventional method originates in assuming that it can 
presume general distribution of a pattern only using the statistic of the model pattern contained in a database. 
[0008] 

[Means for Solving the Problem] In order to solve this problem, in this invention, in addition to the statistic obtained 
from a model set, the statistic which caught change from the model of an input configuration is also made to learn 
beforehand, and is used. Therefore, a model vector covariance input means to input the covariance matrix Cm 
(presumption of the statistical property of a pattern) of a model pattern in this invention. A model-input fluctuation 
covariance input means to make learn beforehand the covariance matrix Cp (statistical information which shows the 
property of change) of the fluctuation to the input configuration which corresponds from each model pattern, and to 
input it Weighted average with a model vector covariance matrix and a model-input fluctuation covariance matrix 
Cs**alpha Cm+(l-alpha) Cp (alpha is the real number of 0< alpha<l) ... (1) 

A covariance weighted average generation means to be alike, to follow and take and to newly generate Matrix Cs. 
and Cs Cs= (AQ 1/2) (Q1/2AT) ... (2) 

(Q the square root matrix of 1/2Q and AT transposed matrix of A) 
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(A is the normalization characteristic vector matrix of Cs) 

It is the matrix DCmDT which changed the first d'lagonalization means and model vector covariance matrix Cm which 
carries out spectral decomposition like (the diagonal matrix which consists of characteristic value to which Q 
corresponds), and obtains matrix Q-1/2AT using matrix D**Q-1/2AT. DCm</SUB>DT=BPBT ... (3) 
(B is the normalization characteristic vector matrix of DCmDT) 

the 2nd diagonalization means which carries out spectral decomposition like (the diagonal matrix which consists of 
characteristic value to which P corresponds), and acquires Matrix B, and these matrix Q-1/2 — AT and B — using 

— H**WBTQ-1/2AT ... (4) 

(W**diag (alphal, atpha2. ... alphan) and (non-negative-number) with alpha i [ suitable ]) are followed. Matrix H is 
generated and held, and it is from the model pattern M and input configuration I to run time. M*'i'*HM, F'MeHI ... (5) 
It has a feature-extraction means for it to be alike, and to follow and to extract a feature vector. 
[0009] In the pattern recognition and the collating unit of claim 1, the feature vector of an input face is received 
further. ||M' - I'll (||^|| is Euclidean distance) ... (6) 

It has a judgment means to elect the model in which a value has the smallest feature vector as a recognition 

result. 

[0010] Moreover, the pattern recognition and the collating unit of claim 2 are the similarity of the feature vector of a 
model and an input configuration further. (M'-D /(|Mi|li) ... (7) 

((♦ and *) evaluate the inner product of a vector, and |*| according to magnitude-of-a-vector). and this value has a 
judgment means by which an input configuration judges whether it is a thing corresponding to that model by whether 
it is more than constant value. 

[001 1] Change of the input configuration from a model pattern can control to intersect perpendicularly with the 
space which a model set occupies by diagonalizing the covariance Cs weighted-average-ized above and the model 
vector covariance Cm using the diagonalization means 1 and 2. and performing a feature extraction according to the 
acquired matrix H. It is surely matched in the model corresponding to an input by the description of a direction 
which intersects perpendicularly with the space which a model occupies in the last process of recognition and 
collating by this even when the difference between a model and an input configuration is great being disregarded. 
Hereafter, this detailed mechanism is explained. 
[0012] 

[Embodiment of the Invention] A model pattern input means by which invention indicated to claim 1 of this invention 
inputs the model pattern M (it is also called model ** KUTORU). An input configuration input means to input input 
configuration I for recognition (for it to also be called an input vector). A model vector covariance input means to 
input the covariance matrix Cm of a model vector, A model-input fluctuation covariance input means to make learn 
beforehand the covariance matrix Cp of the fluctuation to the input configuration which corresponds from each 
model pattern, and to input it. Weighted average with the model-input fluctuation covariance matrix inputted from 
the model vector covariance matrix inputted from said model vector covariance input means, and said model-input 
fluctuation covariance input means Cs**alpha Cm+(1-alpha) Cp (alpha is the real number of 0< alpha<1) ... (1) 
Matrix Cs of the output of a covariance weighted average generation means to be alike, to follow and take and to 
newly generate Matrix Cs. and said covariance weighted average generation means Cs= (AO 1/2) (Q1/2AT) ... (2) 
(A is the normalization characteristic vector matrix of Cs) 

(Diagonal matrix which consists of characteristic value to which Q corresponds) 

( — Q — the square root matrix of 1/2Q, and AT — transposed-matrix) of A — like — spectral decomposition — 
carrying out — this — matrix D**Q-1 / 2AT (Q-1/2 are the inverse matrix of the square root matrix of Matrix Q) 
The diagonalization means of***^* 1st, and matrix DCmDT which changed the model pattern covariance matrix 
Cm according to Matrix D DCmDT=BPBT ... (3) 
(B is the normalization characteristic vector matrix of DCmDT) 

output Q-1/2 of the 2nd diagonalization means which carries out spectral decomposition like (the diagonal matrix 
which consists of characteristic value to which P corresponds), and acquires Matrix B, and said 1st and 2nd 
diagonalization means — AT and B — using — H**WBTQ-1/2AT ... (4) 

(W**diag (alphal, alpha2. ... alphan) and (non-negative-number) with alpha i [ suitable ]) are followed. Matrix H is 
generated and held, and it is from the model pattern M and input configuration I to the run time of recognition. 
M'**HM, r**HI ... (5) 

alike — following — each — a feature vector — M — ' — 1 — ' — extracting — a feature extraction — a means - 

— said — a feature extraction — a means — having extracted — an input configuration — I — a feature vector — I 

— * — a model — a pattern M — a feature vector — M — * — distance — ||M -I' || (||*|| is Euclidean 

distance) ... (6) 

also has an operation that it can recognize except for a changed part of an input configuration as compared 
with a model pattern, by finding out the model pattern which has a small feature vector, providing a judgment means 
to judge to which model an input configuration corresponds by this (recognition), and making model space and 
fluctuation space intersect perpendicularly. 

[0013] Invention of this invention according to claim 2 is the similarity of the feature vector of a model pattern and 
an input configuration. (M -D /(|Ml|r|) ... (7) 

((* and *) have an operation that it can collate with a model pattern except for a changed part of an input 
configuration, when the inner product of a vector and |*| are evaluated according to magnitude-of-a-vector), and 
this value possesses a judgment means to judge whether that input configuration and model are the same by 
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whether it is more than constant value and makes model space and fluctuation space intersect perpendicularly. 
[0014] (Gestalt of the 1st operation) The gestalt of operation of the 1st of this invention is face image recognition 
equipment which elects the face image which is in agreement with the input face image inputted fr-om the video 
camera etc. from the database of a model image photograph. Hereafter, the pattern recognition and the collating 
unit of this invention are explained using Fig. 1 about the case where it applies to face image recognition. 
[0015] The input configuration Input means 1 is constituted by the video camera which photos an input face, the 
digitizer which changes the analog video signal of a video camera into a digital signal, and the image memory which 
memorizes a digital video signal. The model pattern input means 2 is constituted by the image scanner which scans 
and inputs a model image photograph, and the database which stores the model image photograph inputted from the 
image scanner as a digital image file. The covariance count unit 3 is count equipment which calculates a covariance 
matrix by inputting image data from the input configuration input means 1 and the model pattern input means 2. and 
considering that one image data Is a vector. The feature-extraction unit 4 is count equipment which calculates a 
feature vector by considering that one image data is a vector and carrying out vector conversion by the 
transformation matrix while calculating a transformation matrix from a covariance matrix. The Judgment unit 5 is 
count equipment which calculates the distance and the include angle between feature vectors. These count 
equipments may be constituted using a general-purpose processor, and may be constituted using the processor of 
dedication, such as DSP. 

[0016] By the input configuration input means 1, the model pattern input means 2. the covariance count unit 3, and 
the feature-extraction unit 4. a feature space which makes the space of a model pattern and the space of a model- 
input fluctuation vector orthogonalize is selected as an off-line process, and the input configuration input means 1, 
the model pattern input means 2. the covariance count unit 3. and the Judgment unit 5 perform recognition 
processing on-line. 

[001 7] First, off-line processing is explained. Generally the dimension (for example, dimension of the part which a 
face image occupies in the whole image space) of the space which a pattern occupies in data space is quite small 
(for example. 100 dimensions) compared with the dimension (it is 100,000 dimensions if for example, the number of 
pixels is 100,000) of the space of a basis in many cases. Similarly, the fluctuation vector of a model-input also 
occupies the space of a low dimension in data space. First, the covariance matrix Cm which shows the statistical 
inclination of a model pattern is inputted from a model vector covariance input means. The model pattern inputted 
from the model pattern Input means is used for this covariance matrix Cm. Cm**sigma MMT ... (9) 
(MT takes the transposed matrix of Matrix M, and the sum about all sample models {M].) — It follows, and although 
direct count can be carried out. it is satisfactory even if obtained by other approaches. What is necessary is Just to 
show distribution (covariance) of a model pattern. With this operation gestalt, the model pattern covariance Cm is 
calculated using the model person face image set {M} inputted into the database of the image scanner equipment 
which is the model pattern Input means 2. Here, since it is easy, the mean vector of a model vector is set to 0. 
When that Is not right, M (a set (M) averages Ma (M-Ma)) is used. 

[0018] Moreover, the covariance matrix Cp which shows the statistical property of the difference (M-I) of an input 
configuration and a corresponding model is inputted from a model-input fluctuation covariance input means, and the 
fluctuation inclination of an input is made to learn. This covariance Cp uses the sample group of the input 
configuration inputted from an input configuration input means, and the difference of a corresponding model. 
Cp**slgma(M-I) (M-I) T ... (10) 
(M-I) (T Is the transposed matrix of a matrix (M-I)) 

(sum was obtained — all (M. I) — it takes about a pair.) — it follows, and although direct count can be carried out, it 
may be obtained by other approaches. What is necessary is Just to show distribution (covariance) of a difference 
vector. With this operation gestalt, the model-input fluctuation vector covariance Cp actually inputs an input face 
vector {!) (input configuration), and is calculated according to a formula 10 from a difference with a corresponding 
model face pattern. 

[0019] Count of the above two covariances Cm and Cp is carried out by the common covariance count unit 3. The 
Information on these covariances is sent to the feature- extraction unit 4. 

[0020] In the feature-extraction unit 4. first, the weighted average Cs of two covariances is calculated according to 
a formula 1, and the feature-extraction matrix H is generated and held through coincidence diagonalization of Cs 
and Cm. In the case of image recognition equipment, a feature vector (M'] is extracted from a model image vector 
(M), and the feature-extraction unit holds this. 

[0021] Weighted average of two covariances from which the covariance weighted average generation means in the 
feature-extraction unit 4 was acquired, i.e.. the model vector covariance Cm and the model-input fluctuation 
covariance Cp, Cs^^alpha Cm-Kl -alpha) Cp ... (1) 

(alpha is calculated according to real number) of 0< alpha<1. and generates Matrix Cs. Since alpha needs to 
determine a value according to the property of a video camera or an image scanner and needs to determine an 
optimum value according to the image quality of a model Image etc.. It actually tries recognition and calculates a 
value. For example, what is necessary Is to set initial value of alpha to 0.5, to make it change little by little and Just 
to decide a value so that a recognition rate may improve. 

[0022] Next, it opts for selection of the feature space which was suitable for recognition of a pattern using these 
covariances. i.e.. the concrete mechanism of a feature extraction. Here, the feature extraction fr'om a pattern puts 
projecting the pattern (it considering as N dimension) of a basis on the space (for example.-dlmensional [ K ]. K<N) 
of a lower dimension. Choosing a feature space is choosing K axes of coordinates (vector) which constitute such K- 
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dimensional space and which intersect perpendicularly, and, therefore, it is in charge of applying the linear 

transformation (matrix) constituted by such vector with a feature extraction. For this reason, the space of a model 

pattern and the space of a fluctuation vector are orthogonalized by conversion which diagonalizes Matrix Cs and the 

model vector covariance Cm to coincidence. This principle is as follows. 

[0023] Matrix Cs is the first diagonatization means. Cs= (AO 1/2} (Q1/2AT} ... (2) 

(A is the normalization characteristic vector matrix of Cs) 

(Diagonal matrix which consists of characteristic value to which Q corresponds) 

( — Q — spectral decomposition of the square root matrix of 1/2Q and the AT is carried out like transposed- 
matrix) of A, and matrix D**Q-1/2AT is outputted. 

[0024] On the other hand, Covariance Cm is the 2nd [ after being copied by DCmDT by Conversion D ] 

diagonalization means. DCmDT=BPBT ... (3) 

(B is the normalization characteristic vector matrix of DCmDT) 

Spectral decomposition is carried out like (the diagonal matrix which consists of characteristic value to which P 
corresponds), and Matrix B is outputted. 

[(X)25] A feature-extraction means Is matrix Q-1/2AT which Is these outputs, and based on B. H4'%WBTQ-1/2AT ... 

(4) 

(W**diag {... alphal, alpha2. alphaN) and alphai generate Matrix H according to suitable non-negative-numberX and 

hold this. This matrix H is a matrix which performs a feature extraction, alphai is a multiplier which carries out 

weighting to the description, and it is determined by the approach of calculating an optimum value, trying so that a 

recognition rate may Improve. 

[0026] Here, It Is Matrix L L**BTQ-1/2AT ... (11) 

A definition is given. 

[0027] Matrix H performs conversion of constant twice to each component after application of Matrix L. Now, the 
model vector M and input-vector I are received in Matrix L M'**LM, r**LI ... (12) 

When It is alike, and it follows and application, i.e.. a feature extraction, is performed. Matrix Cs and the model vector 
covariance Cm are each by this conversion L L Cs->Cs'=alphaCm'-Kl -alpha) Cp' =alphaslgma (LM) and (LM) T +(1" 
alpha) sigma(L(M-I)) (L (M-I)) T =alphasigma (LMMTLT) 
+ 1-alphasigma (L(M-I) (M-I) TLT) 

= alphaLCmLT-Kl -alpha) LCpLT =L(alphaCm+(l -alpha) Cp) LT =LCsLT =BTQ-1/2ATAQATAQ-1/2B =E (E is a unit 
matrix) ... (13) 

L Cm->Cm'=sigma (LM) and (LM) T =sigma (BTDM) (MTDTB) 
= BTDCmDTB =BTBPBTB =P ... (14) 

— it Is changed Into a unit matrix E and a diagonal matrix P like. 
[0028] To coincidence, it is also a formula 1 to the model-input fluctuation covariance Cp. E=LCsLT = alphaLCmLT+ 
(1 -alpha) LCpLT =alphaP+(1 -alpha) Cp' Cp'=(E-alphaP)/(1 -alpha) ... (15) 

A diagonal matrix is diagonallzed for (P and the real number of 0< alpha<1 and E are diagonalized for alpha like unit- 
matrix). 

[0029] Conversion by the formula 12 shows that model vector covariance Cm' and model-input fluctuation 
covariance Cp* have a characteristic vector In common more clearly than the step of a formula 1 5. further — the 
characteristic value of a formula 15 to the former — descending order — x1>x2>x3> ... the characteristic value of 
the shaft with which the latter corresponds if >xN (all — a non-negative one) — y1=(l-alphax1)/(1 -alpha). y2=(1- 
alphax2)/(1 -alpha), and ... yN=(1-alphaxN)/(1 -alpha) 

Since it becomes, it Is yN>yN-1> to descending order... It is set to >y1 and the ranking of characteristic value Is 
reversed completely. 

[0030] Since the characteristic value of a covariance matrix shows distribution (square) in the direction of a 
corresponding characteristic vector, i.e.. the breadth of distribution, by conversion of a formula 12. the space which 
a model pattern and a model-input fluctuation vector occupy will share all the shafts of distribution, and the ranking 
of the magnitude of the breadth in shaft orientations will reverse It That Is. It can be said that the space of a model 
pattern and the space of a model Input fluctuation vector intersect perpendicularly. Matrix H Is expanding the 
difference of the breadth of distribution further by each shaft orientations after conversion of Matrix L. and 
emphasizes this orthogonalization. 

[0031] The above is an off-line process, the average fluctuation inclination from the model of an input configuration 
is caught and the concrete mechanism of the feature extraction suitable for recognition is determined by this. In 
the case of pattern recognition (matching with a registered model), it is beforehand Inputted off-line through a model 
pattern Input means 2 by which the input of a model pattern also contains a database facility. 
[0032] At the time of activation of pattern recognition. It is incorporated from a video camera and a feature- 
extraction unit calculates feature-vector V according to r**HI to the new input face image I to which predetermined 
processing was performed. Judgment unit ||M' - Y\\ (||*|| is Euclidean distance) ... (6) 

A model face with the feature vector made into min is chosen from the feature vectors {M*] of the model image 
which the feature-extraction unrt holds, and Is outputted as a recognition result. 

[0033] (Gestalt of the 2nd operation) The 2nd operation gestalt of this Invention is a face image collating unit which 
collates the input image photograph inputted as the input face image inputted from the video camera etc. from the 
image scanner etc.. and judges whether it is in agreement. Hereafter, the case where the pattern recognition and the 
collating unit of this invention are applied to face image collating Is explained using Fig. 1. 
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[0034] The model pattern covariance Cm is calculated using the model person face image set {M} inputted into the 
database of the image scanner equipment which is the model pattern input means 2. The count approach of Cm 
Cm'l't'sigma MMT ... (9) 

(MT takes the transposed matrix of Matrix M, and the sum about all sample models {M].) — it follows. 
[0035] The input configuration input means 1 is constituted by a video camera, a digitizer, and the image memory. 
The model-input fluctuation vector covariance Cp actually inputs an input face vector {1} (input configuration), and is 
calculated by the formula 10 from a difference with a corresponding model face pattern. Count of two covariances 
Cm and Cp is carried out by the common covariance count unit 3 above. The information on these covariances is 
sent to the feature-extraction unit 4. In the feature-extraction unit 4. first, the weighted average Cs of two 
covariances is calculated according to a formula 1, and the feature-extraction matrix H is generated and held 
through coincidence diagonalization of Cs and Cm as explained in full detail previously. The above is off-line 
processing. 

[0036] At the time of collating activation, it is incorporated from a video camera and the feature-extraction unit 4 
calculates feature-vector V to the new input ^ce image I to which predetermined processing was performed 
according to T'l'^HI. An input face image fully controls lighting etc. and enables it to photo the best image. 
[0037] In a face image collating unit, since the model face vector M is inputted from an image scanner at the time of 
activation, the feature-extraction unit 4 extracts feature- vector M' according to M***HM at the time of activation. 
Since a model image inputs a photograph with a scanner, a changed part by the difference of the photography 
conditions of a photograph is inputted as it is. 

[0038] At the judgment unit 5. it is from M* and Y. (M -F) /{|Mllr|) ... (7) 

The inner product of a vector and I'i'l calculate the value defined by magnitude-of-a- vector), and ((* and 'i') output 
how [ with collating right by whether it is more than the constant value that the value defined beforehand ] it is. 
[0039] It will be judged whether the best image photoed on that spot and the image of the photograph containing a 
changed part are inputted, they are mapped to a model and the space which a changed part separated, the amount 
corresponding to the cosine of the include angle between two vectors is calculated, and a photograph is in 
agreement with a person. The description which distinction tends to attach can be chosen and collated by acljusting 
suitably the weight of W at the time of opting for Conversion H. 
[0040] 

[Effect of the Invention] As mentioned above, by this invention, it asks for conversion which makes the space which 
a model pattern occupies, and the space which a model-input fluctuation vector occupies intersect perpendicularly, 
and applies to face image recognition and collating. Since a model-input fluctuation vector is the gap from the model 
pattern with which an input configuration corresponds, if an input and a model are collated in the space where a 
model pattern exists after conversion, it can take and remove this gap. Moreover, any photographs of his face can 
be collated with a person in a high precision from many face images and photographs of his face by asking for the 
conversion which makes a model and a changed part cross at right angles beforehand, carrying out a feature 
extraction using this conversion, and collating with a photograph the face which does not belong to the set of a 
model. 

[0041] therefore, compared with a conventional method, it is markedly alike, highly precise face image recognition 
and collating can be realized, and the effectiveness is very large. 

[Translation done.] 
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DESCRIPTION OF DRAWINGS 



[Brief Description of the Drawings] 

[Drawing 1] It is drawing showing the configuration of the face image recognition and collating unit by this invention. 
[Description of Notations] 

1 Input Configuration Input Means 

2 Model Pattern Input Means 

3 Covariance Count Unit 

4 Feature-Extraction Unit 

5 Judgment Unit 
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[Procedure amendment 1] 

[Document to be Amended] Specification 

[Item(s) to be Amended] Claim 

[Method of Amendment] Modification 

[Proposed Amendment] 

[Claim(s)] 

[Claim 1] The pattern recognition and the collating unit characterized by providing the following. A model pattern 
input means to input the model pattern M (for it to also be called a model vector) An input configuration input 
means to input input configuration I for recognition (for it to also be called an input vector) The model vector 
covariance matrix Cm for which it asked from said model pattern M From the model-input fluctuation covariance 
matrix Op for which it asked from the fluctuation vector which deducted input configuration I which corresponds 
from said model pattern M A feature-extraction matrix operation means to hold beforehand in quest of the matrix H 
for feature extractions which carries out linear transformation so that the distribution space of a model pattern and 
the distribution space of a fluctuation vector may intersect perpendicularly mostly. A feature-extraction means to 
extract feature- vector M' from the product of said model pattern M and said matrix H. and to extract feature- 
vector r from the product of said input configuration I and said matrix H at the time of recognition activation. A 
judgment means to Judge to which model an input configuration corresponds the model pattern which has a feature 
vector with the smallest distance of said feature-vector I' and said feature-vector M' by the header and this 
[Claim 2] Said feature-extraction matrix operation nfieans is the pattern recognition and a collating unit according to 
claim 1 characterized by diagonalizing the weighted average matrix of said model vector covariance matrix Cm and 
said model-input fluctuation covariance matrix Cp, and searching for said matrix H. 

[Claim 3] Said feature-extraction matrix operation means is the pattern recognition and a collating unit according to 
claim 2 characterized by searching for the matrix H which projects an input vector on the space which diagonalizes 
the weighted average matrix of said model vector covariance matrix Cm and said model-input fluctuation covariance 
matrix Cp, diagonalizes said model vector covariance matrix Cm, and intersects perpendicularly with the distribution 
space of a fluctuation vector mostly based on those results. 

[Claim 4] Said feature-extraction matrix operation means is a weighted average with said model vector covariance 
matrix Cm and said model-input fluctuation covariance matrix Cp. 
Cs**alpha Cm+(1 -alpha) Cp (alpha is the real number of 0< alpha<l) ... (1) 

The matrix Cs of the output of a covariance weighted average generation means to be alike, to follow and take and 
to newly generate Matrix Cs. and said covariance weighted average generation means 
Cs= (AO 1 /2) (01 /2AT) ... (2) 

(A is the normalization characteristic vector matrix of Cs) 

(Diagonal matrix which consists of characteristic value to which Q corresponds) 
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(Q the square root matrix of 1/2Q and AT transposed matrix of A) 

— spectral decomposition is carried out like and it lines up from this 
D**Q-1/2AT (Q-1/2 are the inverse matrix of the square root matrix of Matrix Q) 
The diagonalization means of ****** 1st, 

The matrix DCmDT which changed the model pattern covariance matrix Cm according to Matrix D 
DCmDT=BPBT ... (3) 

(B is the normalization characteristic vector matrix of DCmDT) 

(Diagonal matrix which consists of characteristic value to which P corresponds) 

** — the 2nd diagonalization means which carries out spectral decomposition like and acquires Matrix B. 
output Q-1/2 of said 1st and 2nd diagonalization means — use AT and B 
H**WBTQ<SUP>-1/2AT ... (4) 

(W**diag (alphal. alpha2, ... alphan) and (alphai are suitable non-negative-number)) 

The pattern recognition and the collating unit according to claim 2 characterized by having a means for it to be alike, 
and to follow, and to generate and hold Matrix H. 

[Claim 5] Said judgment means is the similarity of the feature vector of a model pattern and an input configuration. 
(M -n /(|Ml|r|) ... (7) 

(*-*) (the inner product of a vector and |*| are the magnitude of a vector) 

The pattern recognition and the collating unit according to claim 1 characterized by being alike, following and 

evaluating and judging whether this value of that input configuration and model is the same by whether it is more 

than constant value. 

[Procedure amendment 2] 

[Document to be Amended] Specification 

[Item(s) to be Amended] 0010 

[Method of Amendment] Modification 

[Proposed Amendment] 

[0010] Moreover, the pattern recognition and the collating unit of claim 5 are the similarity of the feature vector of a 
model and an input configuration further. 
(M'-D /(|Ml|r|) ... (7) 

(*-*) (the inner product of a vector and |*| are the magnitude of a vector) 

It is alike, and follows and evaluates and this value has a judgment means by which an input configuration judges 

whether it is a thing corresponding to that model by whether it is more than constant value. 

[Procedure amendment 3] 

[Document to be Amended] Specification 

[Item(s) to be Amended] 0012 

[Method of Amendment] Modification 

[Proposed Amendment] 

[0012] 

[Embodiment of the Invention] A model pattern input means by which invention indicated to claim 1 of this invention 
inputs the model pattern M (it is also called a model vector). An input configuration input means to input input 
configuration I for recognition (for it to also be called an input vector). From the model vector covariance matrix for 
which it asked from said model pattern M. and the modeHinput fluctuation covariance matrix for which it asked from 
the fluctuation vector which deducted input configuration I which corresponds from said model pattern M A feature- 
extraction matrix operation means to hold the distribution space of a model pattern, and the distribution space of a 
fluctuation vector beforehand in quest of the matrix H for feature extractions orthogonalized mostly. A feature- 
extraction means to extract feature-vector M* from the product of said model pattern M and said matrix H. and to 
extract feature-vector Y from the product of said input configuration I and said matrix H at the time of recognition 
activation. The model pattern which has a feature vector with the smallest distance of said feature-vector Y and 
said feature-vector M* A header. It has an operation of recognizing except for a changed part of an input 
configuration as compared with a model pattern, by providing a judgment means to judge to which model an input 
configuration corresponds, and making model space and fluctuation space intersect perpendicularly by this. In 
pattern recognition and a collating unit according to claim 1, said feature-extraction matrix operation means 
diagonalizes the weighted average matrix of said model vector covariance matrix Cm and said model-input 
fluctuation covariance matrix Cp. and invention indicated to claim 2 of this invention searches for said matrix H, and 
has an operation of computing the transformation matrix which makes model space and fluctuation space 
intersecting perpendicularly using a weighted average matrix. Invention indicated to claim 3 of this invention is set to 
pattern recognition and a collating unit according to claim 2. Said feature-extraction matrix operation means The 
weighted average matrix of said model vector covariance matrix Cm and said model-input fluctuation covariance 
matrix Cp is diagonalized. Diagonalize said model vector covariance matrix Cm, and it is based on those results. The 
matrix H which projects an input vector on the space which intersects perpendicularly with the distribution space of 
a fluctuation vector mostly is searched for, and it has an operation of computing the transformation matrix which 
projects a model pattern on space without the effect of a fluctuation vector. Setting invention indicated to claim 4 
of this invention to pattern recognition and a collating unit according to claim 2, said feature-extraction matrix 
operation means is a weighted average with said model vector covariance matrix Cm and said model-input 
fluctuation covariance matrix Cp. 

Cs**alpha Cm-Kl-alpha) Cp (alpha is the real number of 0< alpha<1) ... (1) 
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The matrix Cs of the output of a covariance weighted average generation means to be alike, to follow and take and 
to newly generate Matrix Cs. and said covariance weighted average generation means 
Cs= (AQ 1/2) (Q1/2AT) ... (2) 

(A is the normalization characteristic vector matrix of Cs) 

(Diagonal matrix which consists of characteristic value to which Q corresponds) 
(Q the square root matrix of 1/2Q and AT transposed matrix of A) 
** — spectral decomposition is carried out like and it lines up from this 
D**Q-1/2AT (Q-1/2 are the inverse matrix of the square root matrix of Matrix Q) 

The diagonalization means of ****** 1st, and the matrix DCmDT which changed the model pattern covariance 
matrix Cm according to Matrix D 
DCmDT=BPBT ... (3) 

(B is the normalization characteristic vector matrix of DCmDT) 

(Diagonal matrix which consists of characteristic value to which P corresponds) 

** — output Q-1/2 of the 2nd diagonalization means which carries out spectral decomposition like and acquires 
Matrix B, and said 1st and 2nd diagonalization means — use AT and B 
H**WBTQ-1/2AT ... (4) 

(W**diag (alphal. alpha2. ... alphan) and (alphai are suitable non-negative-number)) 

It has a means for it to be alike, and to follow, and to generate and hold Matrix H, and has an operation of generating 

the matrix H which separates model vector space more strongly from the distribution space of a fluctuation vector 

using Matrix W based on the result of coincidence diagonalization of a model pattern covariance matrix and a 

weighted average matrix. 

[Procedure amendment 4] 

[Document to be Amended] Specification 

[Item(s) to be Amended] 0013 

[Method of Amendment] Modification 

[Proposed Amendment] 

[0013] Invention of this invention according to claim 5 is the similarity of the feature vector of a model pattern and 
an input configuration. 

(M'-n /(|Mi|r|) ... (7) 

(♦-*) (the inner product of a vector and |*| are the magnitude of a vector) 

It has an operation of collating with a model pattern except for a changed part of an input configuration, by it being 
alike, following and evaluating, providing a judgment means to judge whether this value of that input configuration 
and model is the same by whether it is more than constant value, and making model space and fluctuation space 
intersect perpendicularly. 

[Translation done.] 
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• • ■ 

yN= (1— axf^) / (l-a) 
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